Tianfan Xue

 

Tianfan Xue
Vice Chancellor Assistant Professor
the Department of Information Engineering
the Chinese University of Hong Kong
Email: loading... | loading...
Note: tfxue@mit.edu and tianfan@google.com are no longer active, please contact me through emails above
Google scholar | Linkedin | ORCID

Room 717 SHB
The Chinese University of Hong Kong,
Shatin, N.T.
Hong Kong

Biography

Tianfan Xue started as an Vice Chancellor assistant professor at the Department of Information Engineering, the Chinese University of Hong Kong in Novermber of 2022. If you are excited about computational photography, computer vision, computer graphics, and machine learning, please contact for potential full-funded Ph.D., master, postdoc, R.A., and intern positions available.

Tianfan Xue was working in Google Research from 2017 to 2022. Before that, Tianfan Xue received his Ph.D. from MIT Computer Science and Artificial Intelligence Laboratory, under the supervision of William T. Freeman, with thesis committee members Prof. Frédo Durand, Dr. Richard Szeliski, and Dr. Ce Liu. Also, Tianfan Xue received his Master of Philosophy in the Information Engineering Department, The Chinese University of Hong Kong in 2011, under the supervised of Prof. Xiaoou Tang and Prof. Jianzhuang Liu. Tianfan Xue also received my B.E. degree in the Computer Science and Technology Department from Tsinghua University, China, in 2009. Here is Tianfan Xue's Curriculum Vitae [PDF].

Research Highlights

Lab members


Xiao Chen 陈骁
Ph.D. candidate

Xin Cai 蔡昕
Ph.D. candidate

Lihe Ding 丁立鹤
Ph.D. candidate

Zhiyuan You 尤志远
Ph.D. candidate
(Co. w. Chao Dong)

Hongyu Deng 邓宏宇
Ph.D. candidate
(Co. w. He Chen)

Ruiyuan Lv 吕瑞源
Ph.D. candidate
(Co. w. Jiangmiao Pang)

Yutian Chen 陈羽田
Ph.D. candidate

Ruikang Li 李睿康
Ph.D. candidate

Linhua Huang 黄霖华
Ph.D. candidate

Jiarui Wu 吴佳锐
Ph.D. candidate

Visiting Students

Alumni

Selected Publications


PhoCoLens: Photorealistic and Consistent Reconstruction in Lensless Imaging

Xin Cai, Zhiyuan You, Hailong Zhang, Wentao Liu, Jinwei Gu, Tianfan Xue
Conference on Neural Information Processing Systems (NeurIPS), 2024.
Spotlight, [PDF] [Webpage]

Goal Conditioned Reinforcement Learning for Photo Finishing Tuning

Jiarui Wu, Yujin Wang, Lingen Li, Zhang Fan, Tianfan Xue
Conference on Neural Information Processing Systems (NeurIPS), 2024.

 

AdaptiveISP: Learning an Adaptive Image Signal Processor for Object Detection

Yujin Wang, Xu Tian yi, Zhang Fan, Tianfan Xue, Jinwei Gu
Conference on Neural Information Processing Systems (NeurIPS), 2024.

 

HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation

Zhenzhi Wang, Yixuan Li, Yanhong Zeng, Youqing Fang, Yuwei Guo, Wenran Liu, Jing Tan, Kai Chen, Bo Dai, Tianfan Xue, Dahua Lin
Conference on Neural Information Processing Systems (NeurIPS), 2024.

DualDn: Dual-domain Denoising via Differentiable ISP

Ruikang Li, Yujin Wang, Shiqi Chen, Fan Zhang, Jinwei Gu, Tianfan Xue
European Conference on Computer Vision (ECCV), 2024. [Webpage]

 

Event-Based Motion Magnification

Yutian Chen*, Shi Guo*, Fangzheng Yu, Feng Zhang, Jinwei Gu, Tianfan Xue
European Conference on Computer Vision (ECCV), 2024. [PDF] [Webpage] [Code] [YouTube] [Bilibili]

Descriptive Image Quality Assessment in the Wild

Zhiyuan You, Jinjin Gu, Zheyuan Li, Xin Cai, Kaiwen Zhu, Chao Dong, Tianfan Xue
arXiv 2405.18842, 2024. [Project] [Webpage] [PDF]

Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models

Zhiyuan You*, Zheyuan Li*, Jinjin Gu*, Zhenfei Yin, Tianfan Xue, Chao Dong
European Conference on Computer Vision (ECCV), 2024. [Webpage] [PDF]

 

TimeLens-XL: Real-time Event-based Video Frame Interpolation with Large Motion

Yongrui Ma, Shi Guo, Yutian Chen, Tianfan Xue, Jinwei Gu
European Conference on Computer Vision (ECCV), 2024. [Webpage]

 

AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion

Yitong Jiang, Zhaoyang Zhang, Tianfan Xue, Jinwei Gu
European Conference on Computer Vision (ECCV), 2024. [PDF] [Webpage]

Bilateral Guided Radiance Field Processing

Yuehao Wang, Chaoyi Wang, Bingchen Gong, Tianfan Xue
ACM SIGGRAPH, 2024. [PDF] [Webpage] [Code] [Data]

SIGGRAPH Honorable Mention

LenslessFace : An End-to-End Optimized Lensless System for Privacy-Preserving Face Verification

Xin Cai, Hailong Zhang, Chenchen Wang, Wentao Liu, Jinwei Gu, Tianfan Xue
arXiv 2406.04129, 2024. [PDF] [Webpage] [Code]

 

GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction

Xiao Chen, Quanyi Li, Tai Wang, Tianfan Xue, Jiangmiao Pang
Computer Vision and Pattern Recognition (CVPR), 2024. [PDF] [Webpage] [YouTube]

 

Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D priors

Lihe Ding*, Shaocong Dong*, Zhanpeng Huang, Zibin Wang, Yiyuan Zhang, Kaixiong Gong, Dan Xu, Tianfan Xue
Computer Vision and Pattern Recognition (CVPR), 2024. [PDF] [Webpage]

 

Interactive3D: Create What You Want by Interactive 3D Generation

Shaocong Dong*, Lihe Ding*, Zhanpeng Huang, Zibin Wang, Tianfan Xue, Dan Xu
Computer Vision and Pattern Recognition (CVPR), 2024. [PDF] [Webpage] [YouTube]

 

HDRFlow: Real-Time HDR Video Reconstruction with Large Motions

Gangwei Xu*, Yujin Wang*, Jinwei Gu, Tianfan Xue, Xin Yang
Computer Vision and Pattern Recognition (CVPR), 2024. [PDF] [Webpage]

 

EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI

Tai Wang*, Xiaohan Mao*, Chenming Zhu*, Runsen Xu, Ruiyuan Lyu, Peisen Li, Xiao Chen, Wenwei Zhang, Kai Chen, Tianfan Xue, Xihui Liu, Cewu Lu, Dahua Lin, Jiangmiao Pang
Computer Vision and Pattern Recognition (CVPR), 2024. [PDF] [Webpage] [Code&Data]

 

Obj-NeRF: Extracting Object NeRFs from Multi-view Images

Zhiyi Li, Lihe Ding, Tianfan Xue
arXiv 2311.15291, 2023. [PDF] [Webpage]

 

AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training

Yifan Jiang, Peter Hedman, Ben Mildenhall, Dejia Xu, Jonathan T. Barron, Zhangyang Wang, Tianfan Xue
Computer Vision and Pattern Recognition (CVPR), 2023. [PDF] [Webpage]

 

Fast and High-Quality Image Denoising via Malleable Convolutions

Yifan Jiang, Bartlomiej Wronski, Ben Mildenhall, Jonathan T. Barron, Zhangyang Wang, Tianfan Xue
European Conference on Computer Vision (ECCV), 2022. [PDF] [Webpage]

 

How to Train Neural Networks for Flare Removal

Yicheng Wu, Qiurui He, Tianfan Xue, Rahul Garg, Jiawen Chen, Ashok Veeraraghavan, Jonathan T. Barron
International Conference on Computer Vision ICCV, 2021. [PDF] [Webpage]

 

Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image

Shumian Xin, Neal Wadhwa, Tianfan Xue, Jonathan T. Barron, Pratul P. Srinivasan, Jiawen Chen, Ioannis Gkioulekas, Rahul Garg
International Conference on Computer Vision ICCV, 2021. [PDF] [Code]

 

Learned Dual-View Reflection Removal

Simon Niklaus, Xuaner Cecilia Zhang, Jonathan T. Barron, Neal Wadhwa, Rahul Garg, Feng Liu, Tianfan Xue
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021. [PDF] [Webpage]

 

Joint Bilateral Learning for Real-time Universal Photorealistic Style Transfer

Xide Xia, Meng Zhang, Tianfan Xue, Zheng Sun, Hui Fang, Brian Kulis, Jiawen Chen
European Conference on Computer Vision (ECCV), 2020. [arxiv-preprint] [YouTube] [Dataset]

Real-time Localized Photorealistic Video Style Transfer.

Xide Xia, Tianfan Xue, Wei-sheng Lai, Zheng Sun, Abby Chang, Brian Kulis, Jiawen Chen, 2020.
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021. [PDF]

 

Handheld Mobile Photography in Very Low Light

Orly Liba, Kiran Murthy, Yun-Ta Tsai, Timothy Brooks, Tianfan Xue, Nikhil Karnad, Qiurui He, Jonathan T. Barron, Dillon Sharlet, Ryan Geiss, Samuel W. Hasinoff, Yael Pritch, Marc Levoy
ACM SIGGRAPH Asia, 2019, Oral
[PDF] [Webpage]

 

Multi-frame stereo matching with edges, planes, and superpixels,

Tianfan Xue, Andrew Owens, Daniel Scharstein, Michael Goesele, Richard Szeliski
Image and Vision Computing (IVC), 2019
[PDF] [Webpage]

 

Unprocessing Images for Learned Raw Denoising,

Tim Brooks, Ben Mildenhall, Tianfan Xue, Jiawen Chen, Dillon Sharlet, Jonathan T. Barron
Computer Vision and Pattern Recognition (CVPR), 2019
[PDF] [Website] [github]

 

Stereoscopic Dark Flash for Low-light Photography,

Jian Wang, Tianfan Xue, Jonathan T. Barron, Jiawen Chen
International Conference on Computational Photography (ICCP), 2019
[PDF] [YouTube]

 

MoSculp: Interactive Visualization of Shape and Time,

Xiuming Zhang, Tali Dekel, Tianfan Xue, Andrew Owens, Qiurui He, Jiajun Wu, Stefanie Mueller, William T. Freeman
User Interface Software and Technology (UIST), 2018
[PDF] [Website] [github]
Press Coverage: Forbes, BBC, MIT (featured as the 9/19 homepage), etc.

 

Video Enhancement with Task-Oriented Flow,

Tianfan Xue, Baian Chen, Jiajun Wu, Donglai Wei, William T. Freeman
International Journal of Computer Vision (IJCV), 2018
[PDF] [arxiv-preprint] [Website] [github] [Dataset]

 

Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling

Xingyuan Sun*, Jiajun Wu*, Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Tianfan Xue, Joshua B. Tenenbaum, and William T. Freeman
Computer Vision and Pattern Recognition (CVPR), 2018
[PDF] [Website] [Dataset] [Github]

 

Best-Buddies Similarity - Robust Template Matching using Mutual Nearest Neighbors,

Shaul Oron, Tali Dekel, Tianfan Xue, William T. Freeman, Shai Avidan,
IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2018
[PDF] [arxiv-preprint]

 

MarrNet: 3D Shape Reconstruction via 2.5D Sketches

Jiajun Wu*, Yifan Wang*, Tianfan Xue, Xingyuan Sun, William T. Freeman, and Joshua B. Tenenbaum
Neural Information Processing Systems (NIPS), 2017
[PDF] [Website]

 

Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks

Tianfan Xue*, Jiajun Wu*, Katherine L. Bouman, and William T. Freeman,
Neural Information Processing Systems (NIPS) 2016, Oral.

Visual dynamics: Stochastic future generation via layered cross convolutional networks

Tianfan Xue*, Jiajun Wu*, Katherine L. Bouman, and William T. Freeman,
IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI) 2018.
[PDF (conference)] [PDF (journal)] [arxiv-preprint] [Website] [Video] [YouTube] [Code(Github)]

 

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Jiajun Wu*, Chengkai Zhang*, Tianfan Xue, William T. Freeman, and Joshua B. Tenenbaum,
Neural Information Processing Systems (NIPS) 2016.
[PDF] [arxiv-preprint] [Website] [YouTube] [Code(Github)]

 

Single Image 3D Interpreter Network

Jiajun Wu*, Tianfan Xue*, Joseph J. Lim, Yuandong Tian, Joshua B. Tenenbaum, Antonio Torralba, and William T. Freeman,
European Conference on Computer Vision (ECCV), 2016, Oral.
[PDF] [Website] [IKEA-dataset] [Keypoint5-dataset] [arxiv-preprint] [Code(Github)]

3D Interpreter Networks for Viewer-Centered Wireframe Modeling

International Journal of Computer Vision (IJCV), 2018. [arxiv-preprint]

 

A Computational Approach for Obstruction-Free Photography

Tianfan Xue, Michael Rubinstein, Ce Liu, William T. Freeman,
ACM SIGGRAPH 2015, Oral.
[PDF] [Youtube] [Website]

 

The Aperture Problem for Refractive Motion

Tianfan Xue, Hossein Mobahi, Fredo Durand, William T. Freeman,
IEEE Computer Society Conference on Computer Vision and Patter Recognition (CVPR) 2015.
[PDF] [Poster] [Video] [Website]

 

Refraction Wiggles for Measuring Fluid Depth and Velocity from Video

Tianfan Xue, Michael Rubinstein, Neal Wadhwa, Anet Levin, Fredo Durand, William T. Freeman,
European Conference on Computer Vision (ECCV) 2014, Oral.
[PDF] [Slides] [Data] [Website] [Presentation] [Code]

 

Example-Based 3D Object Reconstruction from Line Drawings

Tianfan Xue, Jianzhuang Liu, Xiaoou Tang,
IEEE Computer Society Conference on Computer Vision and Patter Recognition (CVPR) 2012.
[PDF] [Poster] [Dataset]

 

Symmetric Piecewise Planar Object Reconstruction from a Single Image

Tianfan Xue, Jianzhuang Liu, Xiaoou Tang, (CVPR) 2011.[PDF]

3D Modeling from a Single View of a Symmetric Object

Tianfan Xue, Jianzhuang Liu, Xiaoou Tang, Transactions on Image Processing (TIP) 2012. [PDF]

 

Object Cut: Complex 3D object reconstruction through line drawing separation

Tianfan Xue, Jianzhuang Liu, Xiaoou Tang,
Computer Society Conference on Computer Vision and Patter Recognition (CVPR) 2010. [PDF]

 

Joint Example-based DepthMap Super-Resolution

Yanjie Li, Tianfan Xue, Lifeng Sun, Jianzhuang Liu,
International Conference on Multimedia Expo (ICME) 2012. [PDF] [Code]

 

Fast Frame-rate Up-conversion of Depth Video via Video Coding

Yanjie Li, Lifeng Sun, Tianfan Xue,
ACM Multimedia (MM) 2011. [PDF]

* indicates equal contribution

Education

Services

MISC